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Abstract 


There is strong theoretical support for oral reading fluency (ORF) as an essential building 
block of reading proficiency. The current and standard ORF assessment procedure 
requires that students read aloud a grade-level passage (-250 words) in a one-to-one 
administration, with the number of words read correctly in 60 seconds constituting their 
ORF perfonnance. The current study was part of a larger project to develop and validate 
a computerized ORF assessment system - Computerized Oral Reading Evaluation 
(CORE) - to reduce limitations in current ORF measures and procedures. The purposes 
of this technical report are to: (a) document whether the CORE system was accessible 
and useful for teachers, (b) explore potential differences between CORE and traditional 
ORF (i.e., easyCBM) passages, and (c) identify potential deficits in the three CORE 
lengths (~ 25, 50, or 85 words). This information contributes to the response-process 
evidence for the CORE system’s validity. Our results suggest that delivering, scoring, 
and storing ORF assessments online may be feasible, and desirable, for classroom 
teachers across Grades 2 through 4. In addition, although there were no distinct 
differences between CORE and traditional ORF passages, teacher reports suggest that the 
CORE short passages are most appropriate for Grade 2 students while CORE long and 
medium passages are preferred by teachers for students in Grades 3 and 4. 
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Teacher Survey of the Accessibility and Text Features of the 
Computerized Oral Reading Evaluation (CORE) 

As the most prevalent reading assessment in response-to-intervention (RTI) models, 
measures of oral reading fluency (ORF) are used in elementary schools across the country to 
universally screen for students at risk of low reading proficiency, to ensure students are meeting 
teacher expectations, to monitor progress of students receiving reading intervention or identified 
as having a disability, and to predict year-end performance on large-scale reading achievement 
tests. The current and standard ORF administration requires that students read aloud a grade- 
level passage, with the number of words read correctly in 60 seconds constituting their ORF 
performance score, reported as words correct per minute (wcpm). 

There is strong theoretical support for ORF as an essential part of reading proficiency. 
Assessing reading fluency is critical because it functions as an indicator of comprehension and 
overall reading achievement (e.g., Deno, 1985; Fuchs, Fuchs, & Maxwell, 1988; Hosp & Fuchs, 
2005; Pinnell et ah, 1995). Research also indicates that the measurement of ORF growth within 
or across years is important to predict future reading outcomes (e.g. Baker et al., 2008; Kim, 
Petscher, Schatschneider, & Foonnan, 2010; Speece & Ritchey, 2005). 

The current study was part of a larger project to develop and validate a computerized 
ORF assessment system - Computerized Oral Reading Evaluation (CORE) - to reduce 
limitations in current ORF measures. The innovations of our proposed assessment system are 
intended to provide more reliable and valid measurement of students’ ORF for universally 
screening students at risk of poor reading outcomes and monitoring the progress of those 
identified. Our proposed CORE assessment system will change both existing ORF assessment 
and delivery to increase efficiency and improve accuracy of measurement. Traditional ORF 
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measures use long passages (approximately 250 words), in which all sentences are sequentially 
dependent, such that students’ reading is subject to local dependence. To reduce the potential 
local dependence, CORE presents multiple, shorter ORF passages in a single test administration. 
Psychometrically, an added benefit of this approach may be that repeated responses to multiple 
ORF passages will yield less measurement error. CORE passages vary in word length, with three 
lengths of passages offered: short ~ 25 words, medium ~ 50 words, and long ~ 85 words. 

In this study, we compared the CORE passages to traditional ORF passages to explore 
whether all CORE passage lengths function adequately across grades. We surveyed teachers’ 
observations of students’ test- taking to explore potential differences between CORE and 
easyCBM passages and potential deficits in the three CORE passage lengths. The teacher survey 
was based on the work of Hiebert and colleagues (e.g., Fitzgerald et al., 2015; Mesmer, 
Cunningham, & Hiebert, 2012) to help shape the questions about the features (or text 
complexity) of the CORE and traditional ORF passages. The survey was partitioned in two parts: 
(a) feedback on the recordings and software, with questions about the usability of the software, 
the sound quality of the recordings, and the usefulness of the software; and (b) specific feedback 
about the passage type: traditional ORF (easyCBM passage) along with each of the CORE short, 
medium, and long passages. This information contributes to the response-process evidence for 
CORE validity. We hypothesized that all CORE versions would function appropriately; 
however, if an inconsistency was identified for any CORE version, we would consider 
eliminating the problematic CORE version in future system development. 
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Methods 

Sample 

Two local districts that use easyCBM as part of district RTI policy agreed to participate 
in the study for a reduction in the annual cost of the district version of easyCBM. District A 
identified nine teachers (three in each of Grades 2-4) in two schools who agreed to participate in 
the study. In District B, school principals emailed teachers eligible to participate with study 
information, and 10 teachers (three in Grades 2 and 4, and four in Grade 3) in two schools agreed 
to participate in the study. All participating teachers were paid a research incentive ($50 gift 
card) for participation, which required the online scoring of selected passages and the completion 
of a survey. The 19 teachers included 17 females across the three grades, with 1 male in each of 
Grades 2 and 3; no other demographic information was requested from the teachers. 

Measures 

Traditional ORF assessment. We administered the easyCBM (Alonzo, Tindal, Ulmer, 
& Glasgow, 2006) ORF measures as the traditional ORF assessments for comparison to the 
CORE passages. Developed in 2006, easyCBM is an online screening and progress monitoring 
assessment system for use in schools working under an RTI framework, available for an annual 
fee for district-wide adoption. 

The ORF passages used in easyCBM were developed to assess students’ ability to 
fluently read narrative text. During instrument development, each form for a given grade level 
was created to be consistent in length and the readability of each fonn was verified to fit 
appropriate grade-level, initially using the Flesch-Kincaid index feature available on Microsoft 
Word (Alonzo & Tindal, 2008), with further refinement based on field tests with grade-level 
students. The easyCBM assessment system includes 20 alternate ORF passages at each grade 
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level, with three passages specifically identified for use as universal screeners (fall, winter, 
spring) and 17 passages for progress monitoring. The passages were developed to be of 
equivalent difficulty for each grade level following word-count, grade-level guidelines (e.g., 
Flesch-Kincaid readability estimates), and form equivalence empirical testing using repeated 
measures ANOVA to evaluate comparability of fonns (Alonzo & Tindal, 2007). 

These traditional ORF measures have demonstrated features of technical adequacy that 
suggest they are sufficient to meet the needs as the comparative example of an existing 
traditional ORF system. During administration, students are given one minute to read as many 
words as possible in a connected narrative passage of approximately 250 words. The reported 
alternate fonn reliability across Grade 2 forms ranged from .91-. 95 (Anderson, Lai, et ah, 2012). 
Predictive (fall, winter) and concurrent (spring) relations between Grade 2 PRF and spring SAT- 
10 reading scale scores were .59-. 62 and .66, respectively (Jamgochian, et ah, 2010). Grade 2 
PRF performance was strongly correlated (r = .95) with Grade 2 performance on the DIBELS 
ORF measure (Good & Kaminski, 2002; Lai, Alonzo, & Tindal, 2013). The reported alternate 
form reliability across Grade 3 PRF forms ranged from .92-. 96 (Park, Anderson, Alonzo, Lai, & 
Tindal, 2012). Test-retest reliability for Grade 3 PRF ranged from .84-. 94 (Park et ah, 2012). 
Grade 3 PRF perfonnance was strongly correlated (r = .94) with Grade 3 performance on the 
DIBELS ORF measure (Good & Kaminski, 2002; Lai et ah, 2013). Predictive (fall, winter) and 
concurrent (spring) relations between Grade 3 PRF and year-end state achievement reading 
scores were .67, .66, and .67, respectively (Saez et al., 2010). For Grade 4 PRF forms, alternate 
form reliability ranged from .83-. 98 (Alonzo, Lai, Anderson, Park, & Tindal, 2012). Test-retest 
reliability ranged from .86-. 96 (Alonzo et al., 2012). Grade 4 PRF performance was also strongly 
correlated (r = .93) with Grade 4 performance on the DIBELS ORF measure (Good & Kaminski, 
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2002; Lai et al., 2013). Predictive (fall, winter) and concurrent (spring) relations between Grade 
4 PRF and year-end state achievement reading scores were .67, .64, and .66 respectively (Saez et 
al., 2010). 

CORE assessment. We recruited a former teacher to write the CORE passages. The 
teacher held a Master’s degree in counseling psychology and had worked as a high school 
history teacher in California as well as a school counselor, working with students from pre-K 
through high school, in Colorado. In addition, she was part of the team that developed the 
easyCBM reading comprehension and ORF passages, writing 1500-1800 word narrative texts for 
use as reading comprehension passages and 150-350 word narrative texts for use as ORF 
passages across Grades 1-8. She was hired to write the CORE passages primarily because of her 
familiarity with and expertise in passage- writing for this context, and also in recognition of her 
familiarity with the population of students for whom the measures were being written. 

The passage writer was given the following specifications. Each passage was to be an 
original work of fiction, and be ±5 words of the target length (i.e., short = 25, medium = 50, long 
= 85). Each passage was to have a beginning, middle, and end; this broad specification was 
intended to give the passage writer freedom in meeting the word constraint specification, which 
was crucial in this project. The passage was to follow either a problem/resolution fonnat, or 
sequence of events fonnat. The problem/resolution format included: a character, a setting, an 
action, a problem, and a solution or an ending. The sequence of events format included an 
expanded description of a particular setting with a starting event that leads to direct and indirect 
consequences (but not a problem/resolution). The particular setting could be geographical, 
historical, or other special place. In both types of formats, there was a minimum use of dialogue 
and symbols. Exclusion rules for what cannot appear in passages included: religious themes; 



6 


trademark names, places, products; cultural/ethnic depictions; age inappropriate themes (e.g., 
violence, guns, tobacco, alcohol, drugs). The inclusion of character feelings was optional, and 
left to the discretion of the passage writer, given that all other specifications (e.g., exclusions) 
were met. 

Final Year 1 passages included 162 passages total; 54 at each of Grades 2-4, with 9 long 
passages (80-90 words), 15 medium passages (45-55 words), and 30 short ORF passages (20-30 
words). Each passage was an original work of narrative fiction; complete with a beginning, 
middle, and ending; and was written with grade-level appropriate vocabulary and readability 
estimated at the mid-year level for each grade. A single readability score was computed for each 
passage, rounding the average of the Flesch-Kincaid Grade Level (Mac), Spache Readability 
Index (Revised), and Automated Readability Index. All passages were then reviewed by an 
expert in assessment of screening and progress monitoring for errors (e.g., format and 
grammatical), bias (e.g., gender, cultural, religious, and geographical), and alignment with grade- 
level readability. All final readability indices were recorded for each passage; passages with an 
average readability outside the targeted grade level were adjusted to either increase or decrease 
readability, as appropriate. 

Procedures 

Participating teachers were asked to listen to the oral reading recordings from three of 
their students and score each recording using the online system developed for this research study. 
Each student had recordings from the easyCBM PRF passage (=250 words); a long CORE 
passage (=85 words); two medium CORE passages (=50 words each); and three short CORE 
passages (=25 words each). Teachers then responded to a questionnaire intended to elicit 
feedback on the passage audio recordings and software, and provide passage-specific feedback 
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on the short, medium, and long CORE passages, as well as the easyCBM passage. The passage- 
specific items focused on the familiarity of the content type, word decodability, frequency of 
words, the extent to which the passage included words meaningful to students, and the amount of 
text per page (Fitzgerald et ah, 2015; Mesmer et ah, 2012). An example of the passage type 
(easyCBM, CORE long, CORE medium, CORE short) was displayed with the passage-specific 
items. Please see APPENDIX A for questionnaire items. 

The online CORE system was used to administer, score, record, and save all passage 
readings (i.e., both easyCBM and CORE passages). The selected recorded audio files were 
scored by teachers at a later date and in a setting different from that in which the data were 
collected. Teachers listened to each recorded audio file (with the ability to rewind, replay, and 
adjust audio), using the same scoring rules as the traditional ORF procedures. 

Results 

Audio Recordings and Software 

We asked teachers about their experience with the ORF audio recordings and CORE 
software (See Tables 1-4). The majority of teachers across all three grades (16 of 18, 89%) 
reported that accessing the recordings was somewhat easy or very easy, while two Grade 2 
teachers (11%) reported it was somewhat difficult. Nearly all teachers (17 of 18, 94%) reported 
that the sound quality of the recordings was somewhat good or very good ; one Grade 3 teacher 
(6%) reported it was somewhat poor. All teachers reported that having an audio recording of 
their students’ oral reading fluency would be at least somewhat useful for SST or IEP meetings, 
with 15 of 18 (83%) reporting it would be very useful. 
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Most teachers (13 of 18, 72%) reported that scoring a passage online was somewhat or 
much easier than scoring a passage with paper and pencil; two teachers (11%) reported it was 
about the same, and three teachers (17%) reported it was somewhat more difficult. 

Familiarity with Content Type 

To understand students’ familiarity with the content of the passages we asked teachers 
about (a) how closely the passages matched the types of texts used in the classroom for reading 
instructional activities, and (b) the content difficulty of the passage relative to their students’ 
background knowledge (Tables 5 - 12). 

Match to instructional materials. Most teachers (14 of 18, 78%) reported that the 
easyCBM passage was a good or strong match to the types of texts they use in their classroom 
for reading instructional activities (Tables 5 - 8). Similarly, 14 teachers (78%) reported that the 
CORE long passage was a good match while four teachers (22%) reported it was fair (all seven 
Grade 4 teachers reported the CORE long passage was a good match). No teacher reported that 
the easyCBM or CORE long passages were a poor match to classroom reading instructional 
materials. 

Comparatively, 13 teachers (67%) reported that the CORE medium passage was a good 
or strong match to classroom reading instructional texts. Two teachers reported a fair match, a 
majority of Grade 3 teachers reported a poor match. 

Finally, 10 teachers (56%) reported the CORE short passage was a good instructional 
match, four reported it was fair (22%), and four reported it was poor (22%). All six Grade 2 
teachers reported the match was good, suggesting that shorter passages are more grade- 
appropriate for Grade 2. Three of the five Grade 3 teachers reported the CORE short passage was 
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a poor match, one reported it was fair, and one reported it was good. Three Grade 4 teachers 
reported the match was good, three reported it was fair, and one reported the match was poor. 

Match to background knowledge. Teachers were asked about the content difficulty of 
the passage relative to their students’ background knowledge (Tables 9 - 12). Generally, as 
passages got shorter, teachers felt the passages became easier relative to their students’ 
background knowledge. Specifically, 15 teachers (83%) reported the content difficulty of the 
easyCBM passage was about right, one teacher (6%) reported it was somewhat difficult, and two 
teachers (11%) reported it was somewhat easy. 

Similarly, a majority of teachers (12 of 18, 67%) reported the content difficulty of the 
CORE long passage was about right, five reported it was somewhat difficult (28%), and one 
reported it was somewhat easy (6%). Response patterns were fairly similar across grades. No 
teacher reported that the content difficulty of the easyCBM or CORE long passages was either 
very difficult or very easy. 

For the CORE medium passage, 1 1 teachers (61%) reported the content difficulty was 
about right, five reported it was somewhat easy (28%), and two reported it was very easy (11%). 
The majority of teachers in Grades 2 and 4 reported it was about right, however, four of five 
Grade 3 teachers reported it was somewhat or very easy. 

Similar to the reports of instructional match, the CORE short passage was generally rated 
as easier than the other passages. Only seven teachers (none of whom were Grade 3 teachers) 
reported the content difficulty of the CORE short passage was about right (39%), eight reported 
it was somewhat easy (44%), and three reported it was very easy (17%). The majority of Grade 2 
teachers reported it was about right, however, the majority of Grade 3 and 4 teachers reported 
the short passage was somewhat easy. 
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Targeted Readers 

We asked teachers to indicate for which groups of readers the study passages would be 
most appropriate (Tables 13 - 16). The majority of teachers across grades (12 of 18, 67%) and 
within Grades 3 and 4 reported that the easyCBM passage would be most appropriate for 
average readers. The majority of Grade 2 teachers reported that the passage would be most 
appropriate for average or all readers. No teacher reported that the easyCBM passage would be 
most appropriate for struggling readers. 

Half of teachers across grades reported that the CORE long passage would be most 
appropriate for average readers. For Grades 3 and 4, the majority of teachers reported for 
average readers (however, it should be noted all response options were endorsed at Grade 4, 
indicating the variance of responses). Two-thirds of Grade 2 teachers reported the CORE long 
passage would be most appropriate for strong readers, indicating that perhaps these passages 
were viewed as comparably difficult. 

Half of teachers across grades reported that the CORE medium passage would be most 
appropriate for average readers. The majority of Grade 2 and 4 teachers reported that the CORE 
medium passages were most appropriate for average readers, whereas 80% of Grade 3 teachers 
reported that the CORE medium passages were most appropriate for struggling readers, 
suggesting that these teachers viewed these passages as less challenging than the easyCBM and 
CORE long passages. Similarly, half of teachers across grades reported that the CORE short 
passage would be most appropriate for average readers. The majority of Grade 3 and 4 teachers 
reported that the CORE short passages were most appropriate for struggling readers while the 
majority of Grade 2 teachers endorsed the CORE short passages for average readers in their 


classes. 
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Word Difficulty, Decodability, and Frequency 

To understand the characteristics of the words in the study passages, we asked about 
specific word difficulty, word decodability, and word frequency (Tables 17 - 28). 

Word difficulty. We asked teachers whether the passages contained specific words that 
would be difficult for students in their classes (Tables 17 - 20). The majority of teachers (14%) 
agreed that there were specific words in the easyCBM passage that would be difficult for their 
students (all Grade 2 teachers responded this way). In both Grade 3 and 4, two teachers 
disagreed, reporting the easyCBM passage did not contain any specific words that would be 
difficult for their students. 

For the CORE long passage, the majority of teachers (14 of 18, 78%) agreed or strongly 
agreed that the passage contained specific words that would give their students difficulty. This 
result was found for both Grade 2 and Grade 4 teachers; however, while two Grade 3 teachers 
responded similarly, three Grade 3 teachers disagreed that the CORE long passage contained 
specific words that would challenge their students. 

For the CORE medium passage, a majority of teachers overall (11 of 18, 61%) reported 
that the passage did not contain specific difficult words, a result found in Grades 2 and 3 as well. 
However, the majority of Grade 4 teachers agreed that the CORE medium passage contained 
specific words that might be difficult for their students. 

For the CORE short passages, a majority of teachers overall (14 of 18, 78%) reported that 
the passage did not contain specific difficult words. Grade 2 teachers were split; half agreed and 
half disagreed that the passages contained specific challenging words for their students. All 
Grade 3 teachers reported that the short passages did not contain specific difficult words, and two 
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Grade 4 teachers strongly disagreed that the short passages contained difficult words, while only 
one teacher reported the short passages contained difficult words. 

Word decodability. We asked teachers about the grade-level of the decodability of the 
passage words for the average student in their class (Tables 21 - 24). Nearly all teachers across 
grades (17 of 18, 94%) reported that the words in the easyCBM passages were on grade-level 
(one Grade 3 teacher reported the words were above grade-level). 

For the CORE long passages, the majority of teachers within and across grades reported 
that the words were generally on grade-level. Two Grade 2 teachers reported the words were 
above grade-level, and one teacher in each of Grades 3 and 4 reported the words were below 
grade-level . 

Similarly, for the CORE medium passages, the majority of teachers across grades 
reported that the words were generally on grade-level . All Grade 2 teachers and six of seven 
Grade 4 teachers (86%) reported the words were on grade-level ; however, the majority of Grade 
3 teachers reported that the words were below grade-level. 

For the CORE short passages, the majority of teachers across grades, and within Grades 3 
and 4, reported that the words were generally below grade-level. All Grade 2 teachers again 
reported that the words were on grade-level. 

Word frequency. We asked teachers about the frequency of the words in the passages 
relative to average readers in their class (Tables 25 - 28). For the easyCBM passage, a majority 
of teachers (15 of 18, 83%) across all grades reported the words were of moderate frequency. 

One teacher in each grade reported the words were of high frequency. For the CORE long 
passages the majority of teachers across grades (72%), and within Grades 2 and 4, reported the 
words were of moderate frequency. Grade 3 teachers endorsed each of the response options, 
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suggesting less agreement in this group. For the CORE medium passages, half the teachers 
reported the words were of high frequency, but results varied by grade, making interpretations 
tenuous. For the CORE short passages, the majority of teachers across grades (61%), and within 
Grades 3 and 4, reported the words were of high frequency, the majority of Grade 2 teachers 
reported the words were of moderate frequency. 

Text Quantity 

We asked teachers about the amount of text on the screen for each passage (Tables 29 - 
32). A majority of teachers across passages and grades reported that the amount of text was just 
fine. The CORE long passages received almost unanimous support for this response option (17 
of 18, 94%), followed by the CORE medium passages (15 of 18, 83%), the CORE short passages 
(13 of 18, 72%), and finally the easyCBM passages (12 of 18, 67%). Comparing the easyCBM to 
the CORE passages: For the easyCBM passages, the remaining six teachers, evenly split across 
the three grades, reported that rated the amount of text on the screen was too much', whereas for 
the CORE passages (long, medium, or short) the remaining teachers reported that the amount of 
text in the CORE passages was too little for a computer screen. 

Discussion 

One purpose of this survey was to understand whether the CORE system was accessible 
and useful for teachers. Teachers reported that: accessing the ORF audio recordings via the 
teacher-user interface was relatively easy; the sound quality of those recordings was generally 
good; having an audio recording would be useful for SST or IEP meetings; and scoring a passage 
online was easier than scoring a traditional ORF passage with paper and pencil. In addition, 
teachers generally reported that the amount of text presented on the screen by the CORE (and 
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traditional ORF passages) was appropriate. These results suggest that delivering, scoring, and 
storing ORF assessments online may be feasible, even desirable, for classroom teachers. 

The primary purpose of the teacher survey, however, was to explore any potential 
difference between CORE and traditional ORF (i.e., easyCBM) passages, and any potential 
deficits in the three CORE lengths. Survey questions targeted text complexity features of the 
passages, focusing on the familiarity of the content type, word decodability, frequency of words, 
and the extent to which the passage included words meaningful to students. It is important to 
note that across survey items, the Grade 3 teachers tended to rate passages as easier than their 
Grade 2 and 4 peers, which may influence some interpretations across grades, described below. 

The reported instructional match slightly decreased as passage length decreased, such that 
the easyCBM and CORE long passages were better matches with the instructional materials for 
older students and the CORE medium and short passages were better matches for younger 
students. However, in general, the majority of teachers reported that the easyCBM and CORE 
passages were a good match to the types of texts teachers use in their classrooms for reading 
instructional activities. 

Relative to their students’ background knowledge, the majority of teachers reported the 
content difficulty was about right for the easyCBM and the CORE long and medium passages, 
and results skewed toward easy for the CORE short passages. Results varied slightly across 
grades and suggested that the content difficulty of the CORE short passages may be better suited 
for Grade 2 students, as other grade teachers rated the passages easier as length decreased. 

Grade 2 teachers generally reported that all passage lengths were appropriate for average 
readers, except the CORE long passages which were most appropriate for strong readers. The 
teachers in Grades 3 and 4 rated the longer passages more appropriate for average readers, and 
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the CORE short passages more appropriate for struggling readers in their class, indicating short 
passages are perceived as perhaps less difficult. Note that there was not unanimous agreement 
for any group of grade-level teachers for any passage length. 

Across passages and grades, teachers reported more difficult words within passages of 
increased length. This pattern may have emerged because shorter passages contain fewer words 
and therefore fewer opportunities to challenge their students. Similarly, as passage length 
decreased, teachers from the upper grades reported the word decodability became easier, whereas 
Grade 2 teachers reported that words in the shorter passages were generally more aligned with 
grade-level decodability. Overall, teachers generally reported that the shorter passages contained 
higher frequency words and viewed them as generally easier than longer passages. This finding 
aligns with the teachers’ rating that the shorter passages were generally easier, despite the fact 
that the readability indices previously indicated about the same level of difficulty across passages 
and relative to grade-level. 

A broad conclusion can be drawn that there are no distinct differences between CORE 
and traditional ORF (i.e., easyCBM) passages. The teachers in our sample appeared to find the 
CORE passages generally appropriate for their students, and the functionality of the software 
suitably accessible for their use. However, our results also suggest that the CORE short passages 
might be most appropriate for Grade 2 students while the CORE long and medium passages 
might be preferred by teachers for students in Grades 3 and 4. Caution is warranted, of course, 
due to the small sample size of this initial field test study. Additional research is needed to draw 


more robust conclusions. 
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I. Recording Software 


Table 1. Response Results for the Question: “Accessing student recordings was 

Responses 



Grade 



Total 
% (n) 


2 

3 


4 

% 

(n) 

% («) 

% 

O) 

Very difficult 

0 

(0) 

0 (0) 

0 

(0) 

0 

(0) 

Somewhat difficult 

33 

(2) 

0 (0) 

0 

(0) 

11 

(2) 

Somewhat easy 

0 

(0) 

20 (1) 

57 

(4) 

28 

(5) 

Very easy 

67 

(4) 

80 (4) 

43 

(3) 

61 

(11) 


Table 2. Response Results for the Question: “The sound quality of the recordings was:” 

Responses 


Grade 


Total 
% (n) 

2 

3 

4 

% (n) 

% (n) 

% (n) 

Very poor 

0 (0) 

0 (0) 

0 (0) 

0 (0) 

Somewhat poor 

0 (0) 

20 (1) 

0 (0) 

6 (1) 

Somewhat good 

17 (1) 

40 (2) 

29 (2) 

28 (5) 

Very good 

83 (5) 

40 (2) 

71 (5) 

67 (12) 


Table 3. Response Results for the Question: “ Having an historical record of your students ’ oral reading 
(for SST/IEP meetings, parent conferences, etc.) would be:” 

Responses 


Grade 


Total 
% (n) 

2 

3 

4 

% (n) 

% (n) 

% (n) 

Not at all useful 

0 (0) 

0 (0) 

0 (0) 

0 (0) 

Somewhat useful 

17 (1) 

0 (0) 

0 (0) 

6 (1) 

Useful 

33 (2) 

0 (0) 

0 (0) 

11 (2) 

Very useful 

50 (3) 

100 (5) 

100 (7) 

83 (15) 


Table 4. Response Results for the Question: “Compared to scoring a passage with paper and pencil, 
scoring a passage online was: ” 

Responses 

% 

2 

(n) 

Grade 

3 

% (n) 

% 

4 

(«) 

Total 
% (n) 

Much more difficult 

0 

(0) 

0 

(0) 

0 

(0) 

0 

(0) 

Somewhat more difficult 

17 

(1) 

40 

(2) 

0 

(0) 

17 

(3) 

About the same 

0 

(0) 

0 

0 

29 

(2) 

11 

(2) 

Somewhat easier 

67 

(4) 

40 

(2) 

29 

(2) 

44 

(8) 

Much easier 

17 

(1) 

20 

(1) 

43 

(3) 

28 

(5) 
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II. Familiarity Content Type 

Table 5. Response Results for the Question: “ How close a match is this [easyCBM] passage to the types 
of texts that you use in your classroom for reading instructional activities?” 


Grade 


2 

3 

4 

Total 


Responses 

% 

(«) 

% 

( 77 ) 

% 

( 77 ) 

% 

( 7 ?) 

Poor Match 

0 

(0) 

0 

(0) 

0 

(0) 

0 

(0) 

Fair Match 

17 

(1) 

20 

(1) 

29 

(2) 

22 

(4) 

Good Match 

33 

(2) 

40 

(2) 

57 

(4) 

44 

( 8 ) 

Strong Match 

50 

(3) 

40 

(2) 

14 

(1) 

33 

(6) 


Table 6. Response Results for the Question: ‘‘How close a match is this [LONG] passage to the types of 
texts that you use in your classroom for reading instructional activities?” 



Grade 




2 

3 

4 

Total 

Responses 

% ( 77 ) 

% ( 77 ) 

% (77) 

% (77) 

Poor Match 

0 (0) 

0 (0) 

0 (0) 

0 (0) 

Fair Match 

33 (2) 

40 (2) 

0 (0) 

22 (4) 

Good Match 

67 (4) 

60 (3) 

100 (7) 

78 (14) 

Strong Match 

0 (0) 

0 (0) 

0 (0) 

0 (0) 


Table 7. Response Results for the Question: ‘‘How close a match is this [MEDIUM] passage to the types 
of texts that you use in your classroom for reading instructional activities?” 



Grade 




2 

3 

4 

Total 

Responses 

% (77) 

% (77) 

% (77) 

% (77) 

Poor Match 

0 (0) 

60 (3) 

0 (0) 

17 (3) 

Fair Match 

17 (1) 

20 (1) 

0 (0) 

11 (2) 

Good Match 

67 (4) 

20 (1) 

100 (7) 

67 (12) 

Strong Match 

17 (1) 

0 (0) 

0 (0) 

6 (1) 


Table 8. Response Results for the Question: ‘‘How close a match is this [SHORT] passage to the types of 
texts that you use in your classroom for reading instructional activities?” 



Grade 




2 

3 

4 

Total 

Responses 

% ( 77 ) 

% ( 77 ) 

% ( 77 ) 

% ( 77 ) 

Poor Match 

0 (0) 

60 (3) 

14 (1) 

22 (4) 

Fair Match 

0 (0) 

20 (1) 

43 (3) 

22 (4) 

Good Match 

100 (6) 

20 (1) 

43 (3) 

56 (10) 

Strong Match 

0 (0) 

0 (0) 

0 (0) 

0 (0) 
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Table 9. Response Results for the Question: “Relative to my students ’ background knowledge, the content 
difficulty of this [easyCBM] passage is: ” 


Grade 

2 3 4 Total 


Responses 

% 

(n) 

% 

(n) 

% 

(n) 

% 

(n) 

Very difficult 

0 

(0) 

0 

(0) 

0 

(0) 

0 

(0) 

Somewhat difficult 

17 

(1) 

17 

(0) 

0 

(0) 

6 

(1) 

About Right 

67 

(5) 

67 

(4) 

100 

(7) 

83 

(15) 

Somewhat easy 

17 

(1) 

17 

(1) 

0 

(0) 

11 

(2) 

Very easy 

0 

(0) 

0 

(0) 

0 

(0) 

0 

(0) 


Table 10. Response Results for the Question: “Relative to my students’ background knowledge, the 
content difficulty of this [LONG] passage is: ” 


Responses 



Grade 



Total 
% (n) 


2 


3 


4 

% 

(n) 

% 

{n) 

% 

(n) 

Very difficult 

0 

(0) 

0 

(0) 

0 

(0) 

0 (0) 

Somewhat difficult 

33 

(2) 

40 

(2) 

14 

(1) 

28 (5) 

About Right 

67 

(4) 

60 

(3) 

71 

(5) 

67 (12) 

Somewhat easy 

0 

(0) 

0 

(0) 

14 

(1) 

6 (1) 

Very easy 

0 

(0) 

0 

(0) 

0 

(0) 

0 (0) 


Table 11. Response Results for the Question: 
content difficulty of this [MEDIUM] passage 

“Relative to my students’ background knowledge, the 
is: ” 

Responses 



Grade 


Total 
% (n) 

2 


3 

4 

% ( n ) 


% (n) 

% (n) 

Very difficult 

0 (0) 


0 (0) 

0 (0) 

0 (0) 

Somewhat difficult 

0 (0) 


0 (0) 

0 (0) 

28 (0) 

About Right 

83 (3) 


20 (1) 

71 (5) 

61 (11) 

Somewhat easy 

17 (1) 


40 (2) 

29 (2) 

28 (5) 

Very easy 

0 (0) 


40 (2) 

0 (0) 

11 (2) 


Table 12. Response Results for the Question: 
content difficulty of this [SHORT] passage is: 

'Relative to my students’ background knowledge, the 


Responses 


Grade 


Total 
% (n) 

2 

3 

4 

% («) 

% (n) 

% in) 

Very difficult 

0 (0) 

0 (0) 

0 (0) 

0 

(0) 

Somewhat difficult 

0 (0) 

0 (0) 

0 (0) 

0 

(0) 

About Right 

83 (5) 

0 (0) 

29 (2) 

39 

(7) 

Somewhat easy 

17 (1) 

80 (4) 

43 (3) 

44 

(8) 

Very easy 

0 (0) 

20 (1) 

29 (2) 

17 

(3) 
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III. Target Readers 

Table 13. Response Results for the Question: “ This [easyCBM] passage would be most appropriate for 
which groups of readers in your class: ” 


Responses 


Grade 


Total 
% (77) 

2 

3 

4 

% (77) 

% (77) 

% (77) 

Straggling readers 

0 (0) 

0 (0) 

0 (0) 

0 (0) 

Average readers 

50 (3) 

80 (4) 

71 (5) 

67 (12) 

Strong readers 

17 (1) 

20 (1) 

14 (1) 

17 (3) 

All readers 

33 (2) 

0 (0) 

14 (1) 

17 (3) 


Table 14. Response Results for the Question: 
which groups of readers in your class: ” 

“This [LONG] passage would be most appropriate for 

Responses 


Grade 


Total 
% (77) 

2 

3 

4 

% (77) 

% (77) 

% (77) 

Straggling readers 

0 (0) 

0 (0) 

14 (1) 

6 (1) 

Average readers 

17 (1) 

80 (4) 

57 (4) 

50 (9) 

Strong readers 

67 (4) 

20 (1) 

14 (1) 

33 (6) 

All readers 

17 (1) 

0 (0) 

14 (1) 

11 (2) 


Table 15. Response Results for the Question: 
which groups of readers in your class: ” 

“This [MEDIUM] passage would be most appropriate for 

Responses 


Grade 


Total 
% (77) 

2 

3 

4 

% (77) 

% (77) 

% (77) 

Straggling readers 

0 (0) 

80 (4) 

29 (2) 

33 (6) 

Average readers 

67 (4) 

20 (1) 

57 (4) 

50 (9) 

Strong readers 

0 (0) 

0 (0) 

0 (0) 

0 (0) 

All readers 

33 (2) 

0 (0) 

14 (1) 

17 (3) 


Table 16. Response Results for the Question: 
which groups of readers in your class: ” 

“This [SHORT] passage would be most appropriate for 

Responses 


Grade 


Total 
% (77) 

2 

3 

4 

% (77) 

% (77) 

% (77) 

Straggling readers 

0 (0) 

60 (3) 

57 (4) 

39 (7) 

Average readers 

83 (5) 

40 (2) 

29 (2) 

50 (9) 

Strong readers 

0 (0) 

0 (0) 

0 (0) 

0 (0) 

All readers 

17 (1) 

0 (0) 

14 (1) 

11 (2) 
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IV. Word Decodability and Word Frequency 

Table 17. Response Results for the Question: “ This [easyCBM] passage contains specific words that 
might be difficult for students in my class: ” 


Responses 


Grade 


Total 
% (77) 

2 

3 

4 

% (77) 

% (77) 

% (77) 

Strongly disagree 

0 (0) 

0 (0) 

0 (0) 

0 (0) 

Disagree 

0 (0) 

40 (2) 

29 (2) 

22 (4) 

Agree 

100 (6) 

60 (3) 

71 (5) 

78 (14) 

Strongly agree 

0 (0) 

0 (0) 

0 (0) 

0 (0) 

Table 18. Response Results for the Question: 

“This [LONG] passage contains specific words that might 

be difficult for students in my class: ” 






Grade 




2 

3 

4 

Total 

Responses 

% (77) 

% (77) 

% (77) 

% (77) 

Strongly disagree 

0 (0) 

0 (0) 

0 (0) 

0 (0) 

Disagree 

0 (0) 

60 (3) 

14 (1) 

22 (4) 

Agree 

83 (5) 

40 (2) 

86 (6) 

72 (13) 

Strongly agree 

17 (1) 

0 (0) 

0 (0) 

6 (1) 

Table 19. Response Results for the Question: 

“This [MEDIUM] passage contains specific words that 

might be difficult for students in my class: ” 






Grade 




2 

3 

4 

Total 

Responses 

% (77) 

% (77) 

% (77) 

% (77) 

Strongly disagree 

0 (0) 

20 (1) 

0 (0) 

6 (1) 

Disagree 

67 (4) 

80 (4) 

29 (2) 

56 (10) 

Agree 

33 (2) 

0 (0) 

71 (5) 

39 (7) 

Strongly agree 

0 (0) 

0 (0) 

0 (0) 

0 (0) 

Table 20. Response Results for the Question: 

“This [SHORT] passage contains specific words that might 

be difficult for students in my class: ” 






Grade 




2 

3 

4 

Total 

Responses 

% (77) 

% (77) 

% (77) 

% (77) 

Strongly disagree 

0 (0) 

0 (0) 

29 (2) 

11 (2) 

Disagree 

50 (3) 

100 (5) 

57 (4) 

67 (12) 

Agree 

50 (3) 

0 (0) 

14 (1) 

22 (4) 

Strongly agree 

0 (0) 

0 (0) 

0 (0) 

0 (0) 
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Table 21. Response Results for the Question: “For average students in my class, the decodability of the 
words in the [easyCBM] passage is: ” 


Grade 


2 

3 

4 

Total 


Responses 

% 

( n ) 

% 

(«) 

% 

( n ) 

% 

(n) 

Below grade level 

0 

(0) 

0 

(0) 

0 

(0) 

0 

(0) 

On grade level 

100 

(6) 

80 

(4) 

100 

(7) 

94 

(17) 

Above grade level 

0 

(0) 

20 

(1) 

0 

(0) 

6 

(1) 


Table 22. Response Results for 

the ( 

Question: 

“For average students 

in my class, the decodability of the 

words in the [LONG] passage 

is: ” 











Grade 







2 


3 


4 

Total 

Responses 

% 

(n) 

% 

(») 

% 

(n) 

% 

(n) 

Below grade level 

0 

(0) 

20 

(1) 

14 

(1) 

11 

(2) 

On grade level 

67 

(4) 

80 

(4) 

86 

(6) 

78 

(14) 

Above grade level 

33 

(2) 

0 

(0) 

0 

(0) 

11 

(2) 


Table 23. Response Results for the Question: 
words in the [MEDIUM] passage is: ” 

“For average students 

in my class, the decodability of the 

Responses 


Grade 


Total 

% ( 77 ) 

2 

3 

4 

% («) 

% ( 77 ) 

% ( 77 ) 

Below grade level 

0 (0) 

60 (3) 

14 (1) 

22 (4) 

On grade level 

100 (6) 

40 (2) 

86 (6) 

78 (14) 

Above grade level 

0 (0) 

0 (0) 

0 (0) 

0 (0) 


Table 24. Response Results for the Question: 
words in the [SHORT] passage is: ” 

“For average students 

in my class, the decodability of the 



Grade 


Total 

% ( 77 ) 


2 

3 

4 

Responses 

% ( 77 ) 

% ( 77 ) 

% ( 77 ) 

Below grade level 

0 (0) 

80 (4) 

86 (6) 

56 (10) 

On grade level 1 00 (6) 

20 (1) 

14 (1) 

44 (8) 

Above grade level 

0 (0) 

0 (0) 

0 (0) 

0 (0) 
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Table 25. Response Results for the Question: “For average students in my class, the words in the 
[easyCBM] passage are: ” 


Responses 

2 

% ( 77 ) 

Grade 

3 

% ( 77 ) 

4 

% ( 77 ) 

Total 

% ( 77 ) 

Low frequency 

0 (0) 

0 (0) 

0 (0) 

0 

(0) 

Moderate frequency 

83 (5) 

80 (4) 

86 (6) 

83 

(15) 

High frequency 

17 (1) 

20 (1) 

14 (1) 

17 

(3) 


Table 26. Response Results for the 

Question: 

“ For average students i 

in my class, the words 

in the 


[LONG] passage are:” 












Grade 







2 


3 


4 

Total 

Responses 

% 

( 77 ) 

% 

( 77 ) 

% 

( 77 ) 

% 

( 77 ) 

Low frequency 

17 

(1) 

40 

(2) 

0 

(0) 

17 

(3) 

Moderate frequency 

83 

(5) 

40 

(2) 

86 

(6) 

72 

(13) 

High frequency 

0 

(0) 

20 

(1) 

14 

(1) 

11 

(2) 


Table 27. Response Results for the Question: “For average students in my class, the words in 
[MEDIUM] passage are: ” 

the 


Grade 




2 3 

4 

Total 

Responses 

% (77) % (77) 

% (77) 

% (77) 

Low frequency 
Moderate frequency 
High frequency 

0 (0) 40 (2) 

50 (3) 0 (0) 

50 (3) 60 (3) 

0 (0) 
57 (4) 
43 (3) 

11 (2) 
39 (7) 
50 (9) 

Table 28. Response Results for the Question: “For average students in my class, the words in 
[SHORT] passage are: ” 

the 


Grade 




2 3 

4 

Total 

Responses 

% (77) % (77) 

% (77) 

% (77) 


Low frequency 0 (0) 20 (1) 0 (0) 6 (1) 

Moderate frequency 67 (4) 0 (0) 29 (2) 33 (6) 

High frequency 33 (2) 80 (4) 71 (5) 61 (11) 
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V. Text Quantity 

Table 29. Response Results for the Question: “ The amount of text in this [easyCBM] passage is 
for a computer screen: ” 


Responses 



Grade 



Total 
% (n) 


2 


3 


4 

% 

(n) 

% 

(n) 

% 

(n) 

Too little 

0 

(0) 

0 

(0) 

0 

(0) 

0 

(0) 

Just fine 

67 

(4) 

60 

(3) 

71 

(5) 

67 

(12) 

Too much 

33 

(2) 

40 

(2) 

29 

(2) 

33 

(6) 


Table 30. Response Results for the Question: 
computer screen: ” 

“The amount of text in 

this / LONG 1 passage is for a 



Grade 



2 

3 

4 Total 

Responses 

% (n) 

% {n) 

% («) % (n) 

Too little 

17 (1) 

0 (0) 

0 (0) 6 (1) 

Just fine 

83 (5) 

100 (5) 

100 (7) 94 (17) 

Too much 

0 (0) 

0 (0) 

0 (0) 0 (0) 


Table 31. Response Results for the Question: 
for a computer screen: ” 

“The amount of text in 

this [MEDIUM1 passage is 



Grade 




2 

3 

4 

Total 

Responses 

% (n) 

% {n) 

% (n) 

% (n) 

Too little 

17 (1) 

20 (1) 

14 (1) 

17 (3) 

Just fine 

83 (5) 

80 (4) 

86 (6) 

83 (15) 

Too much 

0 (0) 

0 (0) 

0 (0) 

0 (0) 


Table 32. Response Results for the Question: 
a computer screen: ” 

“The amount of text in 

this [SHORT] passage 

is for 



Grade 




2 

3 

4 

Total 

Responses 

% (n) 

% (n) 

% (n) 

% (n) 

Too little 

17 (1) 

40 (2) 

29 (2) 

0 (5) 

Just fine 

83 (5) 

60 (3) 

71 (5) 

72 (13) 

Too much 

0 (0) 

0 (0) 

0 (0) 

0 (0) 
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Appendix A 
Teacher Questionnaire 

Note that each teacher answered item 7 (a through g) for each of the study passage types (i.e., 
easyCBM, CORE long, CORE medium, and CORE short). 


1 . Enter your last name: 

2. Please briefly describe the three selected students as readers: 

3. Accessing the student recordings was: 


Very Difficult 

Somewhat Difficult 

Somewhat Easy 

Easy 

4. The sound quality of the recordings was: 

Very Poor 

Somewhat Poor 

Somewhat Good 

Very Good 


5. Having an historical record of your students’ oral reading (for SST/IEP meetings, parent 
conferences, etc.) would be: 


Not at All Useful 

Somewhat Useful 

Useful 

Very Useful 


6. Compared to scoring a passage with paper and pencil, scoring the passage online was: 


Much More 
Difficult 

Somewhat 

Difficult 

About the Same 

Somewhat Easy 

Much Easier 


The items below pertain to the [passage type, with example]. 


7a. How close a match is this passage to the types of texts that you use in your classroom for 
reading instructional activities? 


Poor Match 


Fair Match 


Good Match 


Strong Match 


7b. Relative to my students’ background knowledge, the content difficulty of ti 

lis passage is: 

Very Difficult 

Somewhat 

Difficult 

About Right 

Somewhat Easy 

Very Easy 


7c. This passage would be most appropriate for which groups of readers in your class: 


Struggling Readers 

Average Readers 

Strong Readers 

All Readers in my Class 

7d. This passage contains specific words that mi; 

ght be difficult for students in my class: 

Strongly Disagree 

Disagree 

Agree 

Strongly Agree 


7e. For average students in my class, the decodability of the words in the passage is: 


Below Grade-Level 


On Grade-Level 


Above Grade-Level 


7f. For average students in my class, the words in the passage are: 


Low Frequency Words 


Moderate Frequency Words 


High Frequency Words 


7g. The amount of text in this passage is for a computer screen: 


Too Little 


Just Fine 


Too Much 











